Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add FIPS support #1233

Merged
merged 2 commits into from
Oct 4, 2022
Merged

Add FIPS support #1233

merged 2 commits into from
Oct 4, 2022

Conversation

maroth96
Copy link
Contributor

@maroth96 maroth96 commented Feb 2, 2022

When FIPS mode is enabled, s3cmd raises a ValueError exception with the message 'digital envelope routines: EVP_DigestInit_ex] disabled for FIPS' (see below). This change will make s3cmd compatible with FIPS mode.

$ fips-mode-setup --check
FIPS mode is enabled.

$ s3cmd put ...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  ...
Problem: <class 'ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
S3cmd:   2.2.0
python:   3.6.8 (default, Jan 19 2022, 23:28:49) 
[GCC 8.5.0 20210514 (Red Hat 8.5.0-7)]
environment LANG=en_US.UTF-8
Traceback (most recent call last):
  File "./s3cmd/s3cmd", line 3209, in <module>
    rc = main()
  File "./s3cmd/s3cmd", line 3106, in main
    rc = cmd_func(args)
  File "./s3cmd/s3cmd", line 392, in cmd_object_put
    local_list, single_file_local, exclude_list, total_size_local = fetch_local_list(args, is_src = True)
  File "/home/michaelroth/s3cmd/S3/FileLists.py", line 367, in fetch_local_list
    total_size = _fetch_local_list_info(local_list)
  File "/home/michaelroth/s3cmd/S3/FileLists.py", line 238, in _fetch_local_list_info
    md5 = loc_list.get_md5(relative_file) # this does the file I/O
  File "/home/michaelroth/s3cmd/S3/FileDict.py", line 48, in get_md5
    md5 = Utils.hash_file_md5(self[relative_file]['full_name'])
  File "/home/michaelroth/s3cmd/S3/Utils.py", line 105, in hash_file_md5
    h = md5()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

@maroth96
Copy link
Contributor Author

maroth96 commented Feb 2, 2022

This should fix this issue: #1005

@fviard fviard added this to the 2.3.0 milestone Mar 2, 2022
@fviard
Copy link
Contributor

fviard commented Mar 2, 2022 •
edited
Loading

Thank you for your PR, I never heard about this parameter for hashlib but it is awesome to unblock people in such a case.

I notice here (https://code.djangoproject.com/ticket/28401) that it is only available since python 3.9 or on Fedora/redhat distributions. But still better than nothing.

I would suggest the following improvement to your PR:
As the md5 will be used a lot of times and also in multiple places in the code, you could do something like:

  • Do the try/except just once inside BaseUtils.py. Something like:
from functools import partial
from hashlib import md5_real
try:
   md5 = md5_real()
except Exception as exc:
  # Comment to explain that it is for fips
  try:
    md5_real(usedforsecurity=False)
    md5 = partial(md5_real, usedforsecurity=False)
  except:
      # Raise original issue
      raise exc
  • import md5 from BaseUtils.py and use it directly in the other places of s3cmd

@maroth96
Copy link
Contributor Author

maroth96 commented Mar 4, 2022

Done. md5 is moved to BaseUtils.

@maroth96
Copy link
Contributor Author

maroth96 commented Apr 4, 2022 •
edited
Loading

@fviard Is this change OK for submission?

@fviard fviard modified the milestones: 2.3.0, 2.4.0 Oct 2, 2022
@fviard fviard merged commit 7977018 into s3tools:master Oct 4, 2022
@fviard
Copy link
Contributor

fviard commented Oct 4, 2022

Merged, thank you very much and sorry for the delay. I think that I did not notice that you pushed the requested changes.

Btw, I will modify further the code in another commit, because as the "usedforsecurity" will be available upstream, it will be good to have that as a default case.

@uipatherthangaraje
Copy link

When is the plan to release it? how can I consume this fix? thanks

@rajivml
Copy link

rajivml commented Dec 7, 2022

@fviard glad this is fixed but I couldn't find a release post october 3rd where this fix is available https://github.com/s3tools/s3cmd/releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
feature-request
Projects
None yet
Milestone
2.4.0
Development

Successfully merging this pull request may close these issues.
4 participants
@maroth96 @fviard @uipatherthangaraje @rajivml